chore: Reducing E2E test sharding #15192
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 3 Ignored Deployments
|
|
Thank you for following the naming conventions! 🙏 Feel free to join our discord and post your PR link. |
Current Playwright Test Results Summary✅ 324 Passing - Run may still be in progress, this comment will be updated as current testing workflow or job completes... (Last updated on 05/27/2024 01:23:03am UTC) Run DetailsRunning Workflow All checks on Github Actions Commit: 1e600be Started: 05/27/2024 01:20:30am UTC
|
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
hash my url generate url hash
Retry 1 • Initial Attempt |
3.55% (6)6 / 169 runsfailed over last 7 days |
26.04% (44)44 / 169 runsflaked over last 7 days |
📄 apps/web/playwright/event-types.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Event Types tests -- future user Different Locations Tests can select 'display on booking page' option when multiple organizer input type are present
Retry 1 • Initial Attempt |
0% (0)0 / 174 runsfailed over last 7 days |
9.20% (16)16 / 174 runsflaked over last 7 days |
keithwillcode
commented
May 26, 2024
| @@ -195,7 +195,11 @@ function getThemeProviderProps(props: { | |||
|
|
|||
| const isBookingPageThemeSupportRequired = themeSupport === ThemeSupport.Booking; | |||
|
|
|||
| if ((isBookingPageThemeSupportRequired || props.isEmbedMode) && !props.themeBasis) { | |||
| if ( | |||
| !process.env.NEXT_PUBLIC_IS_E2E && | |||
There was a problem hiding this comment.
This gets rid of hundreds of instances of this message when the E2E suite runs.
keithwillcode
commented
May 26, 2024
| @@ -215,7 +215,7 @@ function getThemeProviderProps({ | |||
| const isBookingPageThemeSupportRequired = themeSupport === ThemeSupport.Booking; | |||
| const themeBasis = props.themeBasis; | |||
|
|
|||
| if ((isBookingPageThemeSupportRequired || isEmbedMode) && !themeBasis) { | |||
| if (!process.env.NEXT_PUBLIC_IS_E2E && (isBookingPageThemeSupportRequired || isEmbedMode) && !themeBasis) { | |||
There was a problem hiding this comment.
This gets rid of hundreds of instances of this message when the E2E suite runs.
dosubot
bot
added
automated-tests
keithwillcode
changed the title
p6l-richard
pushed a commit
to p6l-richard/cal.com-fork
that referenced
this pull request
Jul 22, 2024
* chore: Removing E2E test sharding * Added back 4 shards for main E2E * Bump build machines to use 8vCPUs * Downgrading builds to 2 vCPUs to see time difference * Reduced shards to 2 to see if flakes are reduced * Increased E2E vCPUs * Put builds back to 4 vCPUs since they need it * Updated E2E suite to use 16 vCPUs and no sharding * Not running in parallel just to test * Put back the parallelism * Get rid of themeBasis warning in test suite * Skipping another flaky test * 2 shards * Update e2e.yml
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Removes unnecessary sharding for the smaller E2E test suites. It seems that sharding and parallelism is adding to the flaky problem. Also, having those jobs sharded does nothing for the speed of the test suite.
Mandatory Tasks (DO NOT REMOVE)
How should this be tested?